Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use tensor in window_reverse to avoid precision issue #617

Conversation

SigureMo
Copy link
Member

@SigureMo SigureMo commented Jul 23, 2024

window_reverse 里将 window_size 转换为 Tensor,windows.shape[0]HW 在动转静下,因为是动态 shape,所以都是 Tensor,而 window_size 是 scalar,会触发 scale OP,x / n 会转为 scale(x, 1/n),这会导致精度问题,进而导致计算结果出错

一个简单的复现样例:

x = paddle.to_tensor(25)
y = paddle.to_tensor(35)
out = int(x / (y * y / 7 / 7))
# 0,但应该是 1

暂时避免触发 scale OP,将 window_size 转为 Tensor 以触发 elementwise_div,未来可以考虑加一个 scale_div OP,让 x / n 转为 scale_div(x, n) 以避免精度问题(等之后 @gouzil 有空试试?)

该问题一直存在,只是之前的类型提升是向左 cast 隐藏了这点而已,3.0-beta 改了类型提升机制暴露了这点

另外还修改了下 predict 脚本里因为类型提升机制修改挂掉的问题,加了手动 cast

Copy link

paddle-bot bot commented Jul 23, 2024

Thanks for your contribution!

@LokeZhou LokeZhou merged commit 106cbba into PaddlePaddle:develop Jul 23, 2024
3 checks passed
@SigureMo SigureMo deleted the use-tensor-in-window-reverse-to-avoid-precision-issue branch July 23, 2024 08:22
westfish pushed a commit to westfish/PaddleMIX that referenced this pull request Sep 25, 2024
…#617)

暂时避免触发 scale OP,将 `window_size` 转为 Tensor 以触发
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants